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ABSTRACT 


Information collection is a critical task in a wide range of taetieal and strategic military 
and civilian operations. The ability to gather aecurate information in a timely manner is a 
difficult task, but sharing that information and making it accessible in near real-time 
compounds the challenges. The ability to do all three in an environment where network 
eonneetivity is intermittent or non-existent seemed virtually impossible until recent years. 

Network eonneetivity plays a signifieant role in how we conduct business. Our 
requirements to share information do not change simply because there is little or no 
existing infrastructure in our area of operations. In fact, information sharing can prove to 
be more important in areas where infrastructure is simply nonexistent. It is for this reason 
that the Lighthouse suite of applieations was eonceived. 

This research takes a deeper look at speeific goals and requirements for 
information eolleetion using handheld devices, and identifies a data synehronization 
algorithm and existing network eonneetivity teehnologies that ean be used to implement 
sharing between handheld devices. The recommendations can be used to enhance 
information sharing, promote accuracy of data, and improve the effieaey of information¬ 
gathering teehniques when implemented in austere networking environments. 
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I. INTRODUCTION 


The Common Operational Researeh Environment (CORE) Eab at the Naval 
Postgraduate Sehool (NPS) developed the Eighthouse projeet that allows intelligenee 
analysts and taetieal operators to visualize the battlefield by providing a map of the 
human terrain (Eongley 2010a, 1). The human terrain system is a military intelligenee 
program used to better understand the loeal population (U.S. Army Human Terrain 
System 2014). Understanding the human terrain takes into aeeount soeiology, linguisties, 
politieal seienee, anthropology, and other soeial seienees {Wikipedia 2014a). 

Eighthouse was developed at NPS by two students and has been adapted and 
implemented in several use eases (Honeggar 2014). The inspiration for the project came 
from difficulties faced in data collection and analysis problems encountered in Iraq and 
Afghanistan. It is typical of information collectors to gather large amounts of data during 
normal operations. While the services were able to capture vast amounts of data, much of 
the information was unstructured text or in written reports. This information overload 
made content organization and management difficult. Oftentimes, information was not 
exploited because of time constraints. Eighthouse is designed to capture structured data in 
the field for rapid exploitation by analysts while enabling powerful analytical 
methodologies to be applied against the data to reduce the uncertainty involved in 
decision making (Honeggar 2014). Eighthouse exploits the advanced capabilities of 
computer and mobile technology to improve the process and reduce the time and effort 
required for data collection and analysis. 

Today’s smartphones and tablets afford incredible storage and processing 
capabilities. The various applications that make up Eighthouse have enhanced collection 
methods by enabling personnel to gather pertinent information using predefined forms 
stored on mobile phones. The data collected is stored on the device and can be sent to an 
email address or uploaded to a website. Other tools within the Eighthouse project are 
used to aggregate the data in order to provide more information to intelligence analysts. 
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This research analyzes the current Android-based Lighthouse suite of applications 
that focus on data collection tools currently employed by the system. It determines the 
requirements for implementing an improved mobile application for data collection. These 
guidelines would help in creating an improved application that can be deployed on 
virtually any mobile device platform, as well as most web browsers on any computer 
operating system. Using popular, open-source software development kits (SDK) and 
application programming interfaces (API), the prototype can be developed to detect the 
type of device on which it is running and take advantage of additional screen real estate, 
processing capabilities of larger devices, or computer processors. 

A. MOTIVATION 

Lighthouse is used by several government agencies to improve data collection and 
analysis processes. A current limitation is that Lighthouse currently requires a constant 
broadband connection to upload data. This connection is often not available due to 
geography and used too much battery power. 

Another limitation is that special equipment is required for information to be 
properly shared in those austere networking environments. Wave radios are one means of 
gaining the required network access for collectors to upload their data. Persistent 
Systems^ is one company that provides this type of equipment but each radio can cost up 
to $5,000 per device. Depending on how critical the information being collected is, this 
may be a small price to pay. 

Mobile devices, however, have sensors that could possibly remove the 
requirement for wave radios or other equipment. This research would lead to an 
improvement in the functionality of the overall system as well as a cost savings by 
offering a solution that removes the requirement for specialized equipment. 


1 Persistent Systems offers a Wave Radio product line that has mostly been used by government 
entities, but has expanded to industrial companies. For more information, visit www.persistentsystems.com. 
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B. OBJECTIVES 

One capability that will make Lighthouse more useful is to make it more reliable 
in various wireless networking conditions that occur in real-life, without the requirement 
of specialized military equipment. The ultimate goal of this thesis is to enable database 
synchronization between data collectors in remote environments without Internet or 
Broadband connectivity. Sharing data in those remote environments can enhance data 
collection and analysis operations 

C. RELEVANCE TO THE DEPARTMENT OF DEFENSE 

The original inspiration for development of Lighthouse was to streamline and 
improve data collection and analysis specifically in austere environments such as the 
areas of operation (AOR) encompassed by the Global War on Terror. Lighthouse was 
used in remote regions of Iraq and Afghanistan with special military equipment because 
of the lack of network infrastructure. This enabled sharing of data from the remote sites 
to the rear headquarters. The insights gained through this research are impactful in 
various areas throughout the DOD to include improved collection capabilities without the 
use of specialized equipment, which also results in a significant cost savings to American 
taxpayers. 

D. THESIS OUTLINE 

Chapter I introduced the research topic and objectives. Chapter II describes the 
evolution of Lighthouse from Field Information Support Tool (FIST) into its current 
form. It explains the motivation and goals hoped to be accomplished. It also discusses 
some of the shortcomings of the current implementation of Lighthouse. Chapter III 
describes several approaches that could be used to improve current operations. Chapter 
IV describes the methodology, results and the challenges faced in developing the solution 
and issues with database synchronization, especially when in a disconnected 
environment. Chapter V summarizes this research and future work to be conducted. 
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II. BACKGROUND 


This chapter discusses some baekground information that is helpful for readers 
prior to moving forward. Familiarity with database operations is not eritieal, as most of 
the difficulties involved in database sharing and synchronization are fairly simple 
concepts to understand. Additionally, levels of network eonnectivity and the types of 
applieations reeommended based on conneetivity is explained within this chapter. 
Finally, the some researeh on the FIST is detailed to introduce the origins of Lighthouse 
to provide readers with the proper frame of referenee. 

A. NORMAL DATABASE OPERATIONS 

The vast majority of eurrent database systems employ client-server architectures, 
requiring a connection to a main database stored on a server. The abrupt spike in mobile 
applications and mobile devices in general created a requirement for a eloser look into 
how database operations in the mobile realm differs from previous standards of database 
operations. Since mobile device capabilities have increased so dramatieally, users expect 
the same kind of growth in their ability to aecess pertinent data. For example, users were 
once satisfied with their ability to aecess their bank aceount balances by driving to the 
nearest automated teller maehine (ATM). However, these days, users who cannot obtain 
this information via the Internet on a personal computer (PC) or mobile phone would see 
this as a serious limitation and would likely switch to a bank that offers this eonvenience. 

B, DEGREES OF CONNECTIVITY 

At the advent of the Internet, protocols were developed to deal with many issues 

such as connectivity and data-replication to ensure accessibility and reliability of data, as 

well as synchronization to establish rules to ensure aeeuraey of data when seemingly 

simultaneous updates take effect. For example, the first ATM did not disperse cash, 

rather only received deposits. As their capabilities and popularity increased, ATM 

machines could be eonnected via a dial-up conneetion, digital subscriber line (DSL) or 

ATM eontroller (ATMC) {Wikipedia 2014b). A husband and wife accessing the same 

account via different ATMs at the same time could cause the data to become out of syne 
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with the main account database. Banks had to implement procedures to ensure that there 
were no errors by replicating the account at the start of the transaction and verifying 
whether there was a change in that data prior to making an actual change to the database. 

A slow ATM connection could mean several changes to the account prior to 
completion of a single transaction. This is a well-known race-condition problem solved 
many years ago in the days of mainframe servers. It goes without saying that network 
connectivity is a critical factor for access to data, as well as reliability of that data. Yet, 
we can also see that certain types of operations can tolerate different degrees of 
connectivity. The biggest challenge to connectivity, especially in mobile settings, is that 
one cannot guarantee connectivity at all times, and a drop in connection can cause data 
synchronization issues that must be handled. 

The degree to which a device must remain connected has a significant impact on 
the flexibility and usability of the device. Maintaining “constant” connections to the main 
database can become extremely expensive and cumbersome, and given the current state- 
of -the-art of deployment of mobile networks, it cannot be taken for granted. 

1. Constant Connection 

Mobile applications that require constant connectivity to a database or other 
server can be rendered useless if a connection is unavailable. As a result, that application 
becomes useless to the device user since the application will simply not function properly 
without connectivity. For this reason, developers must be careful to prevent making 
connectivity a constant requirement. It may be appropriate in some cases, but for many 
applications, constant connectivity is not truly required. 

Another challenge with the requirement for a constant connection is the fact that 
service interruptions cannot always be avoided or predicted in advance. So, developers 
must ensure they account for disruption at every stage of any transaction. They must deal 
with actions that must occur after connectivity is re-established in order to correct any 
potential discrepancies caused by the interruption (Terry 2008). 
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Weak Connection 


Weakly connected systems can communicate through low-bandwidth or high 
latency connections. However, there can be a severe impact if connection drops during 
transaction. The ideal algorithms for this environment must be conservative. The 
algorithm introduced in this thesis can be used in this environment because it avoids 
having unnecessary re-transmissions. 

3, Intermittent Connection 

An intermittently connected system takes into account or expects periods during 
which it cannot communicate. The major difference between this and weakly connected 
systems is that applications designed for intermittently connected systems take into 
account the fact that connectivity is not guaranteed (Terry 2008). This means that the 
applications attempt to gather and hold transactions until the next connection is made. In 
order to ensure that transactions are not lost in the event of a connection dying in the 
middle of a transaction or prior to completion of waiting transactions, the application do 
not totally remove transactions that were recently completed. Rather, upon re-connection, 
the application verities that the previous transactions were successful prior to considering 
the transaction complete, and removing them from the pending transactions queue. 

C. DIFFERENT MODELS 

Various models have been developed which take into account the degree of 
connectivity and other requirements of the application involved. As mentioned 
previously, a disruption in connectivity can have significant impact on the consistency of 
data. This section will detail the requirements for replication of data in some common 
models in use today. 

1, Thin-Client with Remote Access to Data 

One of the predominant models used is storing data on a server that will be 
accessed by mobile devices. Figure 1 is the common architecture employed by 
developers and providers to allow their mobile clients access to centralized data. It 
depicts three different types of mobile devices that connect wirelessly to a database 
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server. The actual backend database is not important to the mobile clients. They simply 
want to be able to access their data. The mobile devices do not handle sharing content 
either. In fact, neither is concerned with any connection other than its own (Long 2011). 


Any Back Office 
Database 


Clients 



Any 

Wireless 

Network 


\ 


36 

GPPS 

Tetra 

W^i 









Sync Buffer 



Sync Any Database - Over Any Network - To Any Device * 

Sync Buffer 

Figure 1. Sample with Central Database Server (after MobiForms 2014) 


In implementations where there is no sync buffer (or replica) residing on the local 
device, the application will not be able to run because it cannot locate the data required. 
Other implementations employ a buffer that can cache some data locally. After an initial 
connection between the mobile device and the remote sync buffer, the device creates a 
local replica. This prevents the application from failing to function due to lack of wireless 
connectivity (Terry 2008). As long as a connection exists, the local information can 
remain in sync with the remote data. In case of disconnection, requests for data will 
access the local data. If these devices require the ability to modify the remote database, 
the devices make modifications to the local buffer and post updates to the remote server 
when a connection to the remote data server becomes available. Mobile devices have 
limited battery life, so a constant connection can consume too much power, rendering the 
device useless in environments where recharging a battery can be difficult. The local sync 
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buffer enhances power saving by only requiring a connection when an update must be 
posted, or the mobile device user requests updates from the database (Nori 2007). 

2. Device-to-Master 

In many cases, mobile devices are used for functions that require data from a 
master database or server to be replicated on the local device itself. One common 
example is an iPod or iPad, but the same applies to other brands of mp3 players and 
tablets. These mobile devices range in size or capacity. Users have the ability to modify 
the contents of their iPod. They can do so without having to re-purchase content 
previously purchased. Today, they might load a movie on their device and decide next 
week to put more music on the device instead. This flexibility is one of the features that 
make these devices attractive to users. If they did not have the ability to modify the 
content, users would not feel they are worth the cost. 

Figure 2 provides a visual depiction of this architecture used by iTunes. This is 
likely most familiar example of the device-to-master model, yet media services are not 
the only service requiring replication of data. Other uses for this type of system are 
management of contacts and electronic mail. While the architecture may appear to be the 
same as the previous description, their behavior is very different. The major difference is 
that thin-client systems do not necessarily require mobile devices to store or replicate any 
data (Terry 2008). Those devices simply access remote sites and display information the 
user wants to see. They do not require the device to actually store any of the information 
accessed. This architecture, however, requires that some amount of data is stored on the 
mobile device from the master. 

In this model, one device is the “master” device. This means that one device acts 
as the authority or manager. Oftentimes, when synchronization occurs, it must occur 
between one device and its master. It is not often that updates are done between more 
than two devices at a time. If there are multiple devices that must synchronize, they do so 
by taking turns, synchronizing with the master in a pairwise fashion (Nori, 2007). 


9 




Figure 2. iXunes UML Deployment Diagram (from iXunes 2014a) 


Sticking with the music example, a user would connect their iPod to their 
computer. Xhe computer may or may not have the entire user’s content stored locally. So, 
that computer would connect to the Internet to access their service provider. In the case of 
an iPod, that provider would be Apple's iXunes Store. Xhe iXunes Store maintains the 
user's account, tracking any purchases and content owned or authorized to be loaded onto 
the user's device (iXunes 2014b). Users are given the opportunity to reload their devices 
at will. Also, they can synchronize purchases made by the same user from different 
devices. 

Proper management of user content is critical for providers to maintain loyal 
customers. If users start to experience diminished service because of the providers’ 
practices, the likelihood that those users will seek more reliable service increases. No 
user wants to log in one day and see their entire music library has disappeared. 
Additionally, users want their experience to be enhanced as much as possible. Xherefore, 
the simpler it is for them to perform common tasks without experiencing errors, the 
better. For this reason, providers attempt to make common functions simple, or 
automatic. Xhey allow users to perform many tasks in background processes so that users 
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need not be ineonvenienced. These prineiples apply to virtually all user content, not just 
music. 


3, Publisher-Subscriber 

Publisher-subscriber systems are defined by their behavior. Publishers publish bits 
of information that are broadcasted to their subscribers. Therefore, subscribers subscribe 
to receive specific types of information from specific publishers. Either party can be 
mobile or fixed, wireless or wired devices (Terry 2008). 

Two simple examples of this type of system are sports or financial updates. Sports 
fans might subscribe to receive scores or other information about their favorite team from 
a specific team or sports channel. Stock market investors may wish to receive periodic 
updates from their investment firms, financial institution or other provider. Weather, local 
news and traffic alerts are other forms of publisher-subscriber systems that are extremely 
popular for mobile subscribers using phones, smart watches, GPS, or satellite radios. 

Figure 3 shows a typical publisher/subscriber system. This case specifically 
depicts the Java Message System, a Publisher/Subscriber system that is explained in 
greater detail at either Oracle's JMS site (Oracle 2014) or the JMS Wikipedia page 
{Wikipedia 2014c). Client 1 is the publisher of a topic that Clients 2 and 3 have both 
subscribed. Upon publishing of the content, the topic is delivered to both clients. 



Figure 3. Typical Publisher / Subscriber System (from Oracle 2014) 
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4. 


Peer-to-Peer 


In true peer-to-peer systems, no device acts as the master. Peers can connect to 
and update one-another locally. They need not be connected through a central server or 
even have Internet access. This type of system is ideal in a weak, intermittently connected 
or disconnected environment. This is the major advantage of this type of system; 
however, synchronization of data could be an issue if developers are not careful about 
decentralized conflict resolution (Terry 2008). 

Figure 4 depicts a peer-to-peer local network where mobile devices share 
information in their databases with one another. The database can either be a distributed 
database where each device holds different portions of the entire database. It can also 
represent a system that starts with one database that is replicated on each mobile device 
and updates are replicated to the other devices as needed. 





Figure 4. Peer-to-peer network of mobile devices (after MobiForms 2014) 



5, Ad Hoc Sensors 

The final model we discuss is Ad Hoc Sensors. Sensor networks are mesh 
networks comprised of nodes that gather certain information or data. For example, one 
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might have a network of eameras with motion sensors that start reeording when the 
deviee senses motion. Eaeh node has wireless eonneetions between them and may share 
data between eaeh deviee on the mesh network. Another example eould be temperature 
sensors eonneeted to one another wirelessly, ultimately, providing the information to a 
PC. These deviees eould be used as an advaneed warning system for a severe weather or 
voleano eruptions. Motion sensors with eameras attaehed eould be integrated into a mesh 
network as a wireless fenee and deteet a perimeter breaeh, aetivating the eamera to take 
images. The eaptured images ean be sent to a PC on whieh a person would verify whether 
the breaeh was made by a person, animal or strong gust of wind. 

D, PREVIOUS SOLUTION 

Commereial-off-the-shelf (COTS) solutions to solve teehnieal problems within 
the military are eommonplaee. The military proeurement system ean take several years 
and billions of dollars to develop and implement speeifie military equipment needed for a 
speeific mission. Making slight modifieations to eommereial solutions is one method of 
signifieantly redueing taxpayer costs. 

FIST is a COTS system comprised of commercially available smartphones, 
customized software, and a commercial backend or portal for information management. 
(Longley 2010a) The system was used to enable information collection in remote 
environments using commercial smartphones, right out of the box. This enhanced 
collection efforts by simply making it easier to gather pertinent information, thereby 
increasing the number of reports collected. This increase required more robust means of 
intelligence analysis, since the speed of collection was so significantly improved. 

1. The Field Information Support Tool 

The Field Information Support Tool (FIST) started out as a project to integrate 
cellular technology into the Army’s tactical radio frequency (RF) networks. Fventually, it 
grew into a means of gathering information using a smartphone application and sharing 
that information more readily, in any environment (Fongley 2010a). It was an entire suite 
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of commercial capabilities that was used to improve intelligence operations. It enabled 
intelligence operators to collect, save and forward reports more easily, and allowed 
analysts to view and conduct their analysis in a more timely manner. 

Some areas in which FIST has proven its usefulness include military 
counterinsurgency and counter-IED operations, humanitarian assistance and disaster 
recovery operations, anti-gang as well as anti-drug operations (Longley 2010a). Figure 5 
shows a diagram of FIST Components. FIST incorporates a method for collection of data, 
transferring that data to a central server where intelligence analysts could conduct 
geospatial, link, temporal, and social network analysis, and information exchange with 
external data to include the Combined Information Data Network Exchange (CIDNE) 
and the International Studies of Violent Groups (ISVG). 


FusionView 





accessible 

database 


Eigure 5. Diagram of EIST Components (from Eongley 2010a, 3) 
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a. System Components 

FIST consisted of several eomponents known as Collect, FusionPortal and 
FusionView. 

(1) Collect was an Android-based applieation used to eolleet data in the field 
and transfer data the eentral database. It used forms and a “store and forward” method of 
information delivery. Information was eolleeted in very remote, poorly eonneeted 
environments using smartphones. In the event that network eonneetivity was dropped, 
reports were stored loeally and sent upon re-establishment of eonneetivity. 

(2) FusionView was a system designed by Kestral Teehnology Group (KTG) 
in eonjunetion with the Naval Postgraduate School to aggregate data eolleeted. It 
provided a means of data visualization, asset traeking, dynamie report updating, alerts 
and alarms that eould be viewed on laptops, handhelds or network elusters (Longley 
2010a, 8). 

(3) The FusionPortal, also designed by KTG, provided a means for 
integration with eommereial software for further analysis. It was a web-based 
management system that worked with: UCINET, a soeial network and cultural domain 
analysis tool developed by Analytie Technologies; Pajek, another soeial network analysis 
program that provides visualization and analysis that compliments UCINET; the 
Organizational Risk Analyzer (ORA), is a network assessment tool developed at 
Carnegie Mellon that examines ehanges in networks over time, and identifies key players 
and vulnerabilities; Areinfo, a geospatial analysis tool widely used throughout the 
Department of Defense (DOD) (Longley 2010a, 13). The system also allowed for simple 
integration with the Worldwide Civil Information Database, the International Studies of 
Violent Groups and the Combined Information Data Network Exehange (CIDNE) 
(Longley 2010b). 

b. FIST Goals 

The main reason for development of a “new” system or adopting a new way of 
doing business is generally for a pereeived improvement or savings in resources and/or 
time. FIST was able to aecomplish various improvements. 


15 



(1) Rapid prototyping was one of the reasons for choosing to go with COTS 
equipment rather than purpose-built equipment catered to the army for a specific need. 
The COTS approach has obvious advantages. The main savings was in the cost and time 
it normally takes to develop a military or government hardware solution. By customizing 
some of the existing smartphone applications instead of developing a military 
smartphone, development costs were significantly reduced. This allows for rapid 
adaptation and testing of commercial smartphone applications (Longley 2010a). 

(2) Ease of training was another critical reason for adapting FIST. Many 
military solutions involve time and cost intensive training and maintenance modules. 
Since smartphones have become ubiquitous and users are already familiar with 
smartphone usage in their personal lives, training operators to use a smartphone form is a 
fairly trivial task. Virtually no training has to happen for an operator or information 
gatherer to be able to use a form based application. The most difficult part of FIST 
seemed to be training a “form manager” to create the different forms to be used for 
gathering the critical information. 

(3) Improved data collection and analysis was also accomplished by enabling 
the collected information to be shared easily. That rapid sharing cut down the overall 
analysis time, simply by allowing the analysts to obtain the information collected more 
quickly than usual. The application also enabled the data to be stored or exported in 
several different formats. This automated conversion made it easier for intelligence 
analysts to use other commercial analysis tools in their analysis (Fongley 2010a). 

2, Lighthouse, Version 1,0 

As previously stated, the original concept resulted in the creation of FIST. As the 
tool became more popular, capabilities expanded and other tools were integrated into the 
system, it became more commonly known as the Fighthouse Project. It can be thought of 
as a living and breathing system, as other tools are constantly being researched and 
integrated into the system as needed. 
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a. System Components 

(1) Open Data Kit (ODK) is a free, open-source data collection tool-kit that 
was developed by the University of Washington's Department of Computer Science and 
Engineering (ODK 2014a). It enables organizations to create, implement and manage 
their very own mobile data collection systems. Using the tools within ODK, an 
organization can build their own forms that are loaded onto smartphones or other mobile 
devices. Once loaded, that device can be used to collect virtually any type of data in the 
field, including location information, audio-visual, photos, barcodes, or just old-fashioned 
text data. Of course, none of this is extremely useful without the aggregation tool that is 
also included. According to their website, it uses Google's infrastructure and local servers 
with MySQL and PostgreSQL on the backend (ODK 2014b). 

(2) Google Fusion Tables are used to do basic link analysis and create line, 
bar and pie graphs (Longley 2012a). The data can be exported into comma-separated 
values file format. These visual representations are sent to remote data collectors and 
decision makers. 

(3) Organizational Risk Analyzer (ORA) remains a large part of Lighthouse. 
It has powerful, dynamic network analysis capabilities based on the specific data that is 
uploaded to the server. According to the Lighthouse training website, “ 

ORA’s approach is the idea of a meta-matrix of networks that not only 
includes social networks but knowledge networks (who knows what), 
information networks (what ideas are related to what), assignment 
networks (who is doing what), need networks (what knowledge is needed 
to do the task) and so on. (Longley 2012b) 

The ability to export the information into various file formats makes it easier to 
move information into the different tools and obtain the required data analysis. 

b. Use Cases 

(1) Law Enforcement Agencies have been using Lighthouse tools for counter¬ 
gang and counter-drug operations. Specific information about known and suspected drug 
dealers or gang members enables officers to perform social network analysis that has 
proven invaluable to combat these well organized criminal organizations. Not only is the 
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information viewable on a map in order to overlay different events and people, but it is 
also able to look deeper and create possible links for officers to investigate further. 

(2) Humanitarian Assistance and Disaster Recovery operations can be 
significantly enhanced using Lighthouse. In fact, it was used to collect survey 
information following the earthquake in Haiti, by a team from the University of Central 
Florida Institute of Simulation and Technology. 

(3) Counterinsurgency and counter-IED operations are two areas where the 
urgency of information collection cannot be overstated. The most recent information 
tends to be the most critical, and sharing this information among collectors is also critical. 
Using social network analysis tools and modifying the type of information gathered, 
counter-IED operations are tremendously enhanced. Intelligence personnel can now 
analyze component similarities, attributes and other characteristics that can shed some 
light on the various bomb-making networks in various regions of the world. 
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III. LIGHTHOUSE: THE NEXT GENERATION 


A. GOVERNMENT-OFF-THE-SHELF SOLUTION 

Government-off-the-shelf (GOTS) systems are generally COTS systems that are 
created, modified, developed and owned by the government. Usually these tools are 
developed by the technical staff of government agencies with a requirement to fill. These 
systems can often be shared between different government agencies without additional 
costs involved. This is usually a more inexpensive approach to fulfilling a requirement 
than contracting out a job. Also, using open-sourced programming tools can help 
facilitate programming and development needs. Catering open source programs to suit 
the needs of Lighthouse users could streamline the tool rather effectively. The CORE Lab 
created a working group of Lighthouse users to determine what improvements should be 
made to the current system.^ A GOTS solution might be something to consider. This 
chapter describes many of the ideas discussed during the working group discussions to 
enhance Lighthouse. 

B. DISCONNECTED TOOLS 

In its current form. Lighthouse has several disjointed tools that are used for one 
main purpose. That is, to enhance data collection and intelligence analysis efforts, 
especially in austere environments (Longley 2010a, 1). 

When using the current system, users cannot help but notice that the 
environments, or graphical user interfaces used, vary from one function to another. In 
fact, users cannot easily switch between the different missions or tools within Lighthouse. 
Each tool behaves differently from the other since they were all created by different 
companies and with different goals in mind. With the working group's goals in mind, and 
recognizing that many open-source tools already exist that can enhance operations, there 
is no need to totally re-invent the wheel. Many of these tools can be combined into an 

^ Members of the Lighthouse working group: MAJ Sam Kemokai (USA, SF), CW4 Chad Machiela 
(USA, SF), Ms. Linel McCray (Monterey County Health Department), and Massachusetts State Trooper 
Stephen Gregorcyk 
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updated version of Lighthouse that would give the system the look and feel of an 
integrated system, rather than the disjointed system that is currently in use. 

1. Interface for Form Creation and Integration 

One of the major functions of Lighthouse is the ability to create forms based on 
an agency's needs. Each mission would typically require a different form to collect the 
desired information pertinent to the mission. This is an extremely powerful function. The 
original system used Zerion’s iFormbuilder to view, sort filter and export data for 
analysis (Longley 2011). By 2012, it transitioned to ODK. However, there are two 
fundamental flaws inherent in the way this tool is currently used. 

a. Separate /Disparate Data 

One challenge is that each mission is separate. This means that the database 
created by a specific form is for a specific purpose. This data is held totally separate from 
other databases on the same device. Since these forms create different databases, 
comparing the information gathered on different forms is not a trivial task, although not 
impossible. This also makes it difficult to make even modest adjustments such as adding 
a variable. Comparing data from the two databases simply cannot be done easily on the 
mobile device unless the data is combined into the same, shared database. Every interface 
is designed for a specific mission and integrating data between the different forms is not 
part of the current system design. Comparisons can be done, but it requires a third 
interface. This type of disjointedness is not considered a positive feature. 

b. Stand-Alone Tools 

Another challenge is that because each tool within the Lighthouse Project is an 
actual stand-alone tool, each has a different look and feel. Normally, systems are 
designed with consistency in mind. The look and feel of a system could make or break 
the system. Users prefer when different screens of a specific system look and feel the 
same and natural to the task being performed. That is, buttons and background colors, 
layout and other characteristics should look and behave the same. This would make it 
easier for users of one tool within Lighthouse to feel comfortable with other tools in 
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Lighthouse. Unfortunately, sinee it is a set of tools from different manufaeturers and 
developers, eaeh tool behaves differently and interoperability can be challenging. 
Additionally, some tools perform similar functions but provide results in different file 
formats. One tool that can convert into the desired format would be the ideal solution and 
prevent overlapping capabilities. 

2, Data Sharing without Network Connectivity 

Under normal or ideal conditions, users can access nearly anything they desire as 
long as they have access to the Internet. Unfortunately, connectivity varies greatly from 
one location to the next, especially in environments in which Lighthouse is likely to be 
used. What happens when network connectivity cannot be taken for granted? 

Lighthouse currently functions relatively simply with devices purchased through 
normal commercial mobile telephone retail providers. Non-sensitive data can pass 
through indigenous 2-4G networks or Wi-Fi hotspots if available. However, remote 
locations with limited infrastructure may require special equipment obtained through 
military contracts or other special processes. Without special equipment or additional 
infrastructure providing the much-needed connectivity, although collectors can continue 
to gather information, there is no means of sharing that data. Adding any one of the 
following communications capabilities would allow for collectors to share their 
information and render special equipment unnecessary, saving taxpayers the cost of that 
special equipment. The requirement for commercial or military grade data transfer 
methods depend on user requirements and sensitivity of data being collected. 

a. Near Field Communication (NFC) 

Near field communications is a method of communications that allows transfer of 

information between mobile devices without the use of wires (NFC Forum 2014). This is 

done by placing the devices in very close proximity, such as touching or within three 

inches. Generally, companies implement this through proprietary protocols, suited their 

own purposes, making it difficult for transfer between devices made by different 

manufacturers. There are several modes in which NFC is implemented in order to support 

various functions. The NFC Forum was established to standardize protocols to make 
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"life easier and more eonvenient for consumers around the world by 
making it simpler to make transactions, exchange digital content, and 
connect electronic devices with a touch. NFC is compatible with 
hundreds of millions of contactless cards and readers already deployed 
worldwide." (NFC Forum 2014) 

There are several operating modes which NFC-enabled devices are able to 
support. 

(1) Card Emulation Mode allows NFC enabled devices to interact with 
external readers by simulating smart cards. This enables users to make purchases and 
other transactions with a simple touch. 

Lighthouse collectors could use this technology to provision mission specific 
information in their devices with little to no effort, since this capability is already 
integrated into Android devices. To do so, they would simply modify the settings on their 
devices at the time they wish to share information and enable NFC. Then they would 
select the file they wish to share. Finally, they would touch each device that needs the 
information. 

So, a team leader receives word of a short notice mission and downloads the 
mission pack prior to the start of the mission. The mission pack contains all of the 
relevant information / database for the area of operation. The team of collectors would 
not have to wait for every collector to download the relevant data. Rather, that time could 
be used to perform other essential pre-combat checks. Since using NFC and Beam is so 
simple, the team leader can now touch each team member's device, providing them with 
the relevant database information. Additionally, in other operations that are less 
dangerous, collectors can share their updates with one another when they meet for lunch, 
for example, ensuring that every team member has the most updated information. 

To use this capability with Lighthouse could be beneficial, but does not really do 
enough to enhance Lighthouse capabilities. While this capability may be nice to have, 
having to be within such close proximity is extremely limiting. 

(2) Peer-to-Peer Mode is used for two-way sharing of information. It uses the 
ISO/IEC 18092 standard and is based on the forum’s Logical Link Control Protocol 
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Specification (NFC Forum 2014). It can be used for devices to exchange photos, business 
cards, and even Bluetooth or Wi-Fi parameters. 

This could benefit Lighthouse by making it easier to share connection information 
prior to the start of a mission. Collectors would touch devices in order to establish a list 
of devices to which connections are allowed, essentially creating an access list. By doing 
this, devices could automatically connect to “allowed” devices and automatically decline 
connections to devices not on the access list. These functions could happen in the 
background, enabling the data collector to continue gathering information rather than 
stopping collection efforts to verify a device or establish connectivity in the field. 

(3) Reader/Writer Mode allows enabled devices to read information stored in 
NFC tags that are embedded on posters or other marketing displays. This mode is often 
used for updating frequent flyer miles or tapping to obtain special offers. It is compliant 
with NFC-A, NFC-B andNFC-F schemes (NFC Forum 2014). 

The use of this mode in Lighthouse would be extremely limited. It could prove 
beneficial for inventory tracking and surveys, but is not a capability normally associated 
with information collection with respect to social networking. 

b. Bluetooth 

Bluetooth technology is another wireless standard for connecting to external 
devices. The major difference between Bluetooth and NFC is the range for Bluetooth is 
much greater. Rather than being only inches apart, Bluetooth devices can connect at 
distances up to 100 meters (Bluetooth 2014). Having the ability to share information 
between data collectors would prove to be a significant improvement to Lighthouse. This 
would mean that the information collected could be automatically distributed to nearby 
devices rather simply. These local updates would occur much more quickly than having 
each collector send information to the central server and downloading any changes. Data 
synchronization would be improved significantly using the algorithm described in the 
next chapter. Additionally, this local sharing would mean collectors have nearly 
immediate access to information updates than could be acted upon more readily. 
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While having the ability to share data eolleeted by sending files between deviees, 
it eould prove more powerful to share data in the form of objeets holding speeifie reeords 
instead. For example, the applieation eould be programmed to automatieally eonneet to 
and send data to external deviees based on a speeifie event, as in a user has modifications 
or new records to send. This “automatic” attempt to connect and share data would happen 
without other actions taken by the data collector. It could be based on a specific amount 
of time, for example every fifteen minutes; or it could be based on a user having made a 
certain number of updates. 

c. Wi-Fi Direct 

A third wireless technology in growing in popularity is Wi-Fi Direct. This 
technology would be the most ideal when operating in remote or unreliable networking 
environments because it allows for the most distance between mobile devices that wish to 
share data. This would enable a team of collectors in remote locations to share data with 
one another at faster speeds, up to 250 mbps, and distances up to 200 yards (Malone 
2010 ). 


C. DEVICE INDEPENDENCE 

In most cases, when applications must be changed or upgraded, software must be 
re-written for each type of device on which that code is supposed to run. For example, 
iOS devices cannot run code written for an Android device. Rather, the code must be re¬ 
written and compiled specifically for iOS devices. Several software development kits 
allow developers to create applications that can be run on virtually any mobile device or 
laptop browser, rather than writing different software for each device. Appcelerator and 
PhoneGap are two such software development kits that could be used to create the new 
and improved Lighthouse Application. 

As previously mentioned, the major benefit of using one of these SDKs is that the 

program need only be written once. Theoretically, the code is compiled to run on any 

device. In fact, the user may not be able to notice a difference in how the application runs 

on an iPhone, iPad, Android tablet, or laptop web browser for that matter. The developer 

could choose to make the application look and perform in the exact same manner 
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regardless of the device, or take advantage of additional screen real estate and processing 
power differences between smart phones, tablets, laptops and/or desktops. 

D, PROPOSED CAPABILITIES 

The next generation of Lighthouse proposes many enhancements to the current 
process. One of the ideas from those meetings was to develop an application that would 
access various features of the overall system with the click of a button. Each button or tab 
would serve a specific function and are discussed below. Considering that most of the 
software comprising the Lighthouse Project is either open-source or GOES software, they 
can be consolidated into one application. The user interface can be designed to integrate 
various existing functions and improved upon. 

Android developers use “Fragments” and “Activities” to create different views 
and move seamlessly between different functions of an application. Each fragment can 
behave as an interface for a different set of activities and serve distinct purposes. The 
second generation of Eighthouse could consolidate at least eight different operations for 
its users. These functions could be broken up into several tabs providing better flow and 
function to Eighthouse users. Mike Stevens, manager of the CORE Eab, created some 
possible screens to demonstrate the functions to be incorporated in the next Eighthouse 
version, after meetings with the Eighthouse working group. 

The first activity would be considered the “Home” screen. There would be several 
tabs or buttons that would take the user to another activity of the user’s choosing. This 
screen could look something like the screen depicted in Figure 6. This figure also depicts 
the “Help” screen, which displays helpful information about the application to the user. It 
could also display information about the mission requiring the information being 
collected. The highlighted or bolded tab is the active screen or activity. On the home 
screen, a user could select the mission by changing the settings, ultimately loading the 
mission database on to the remote device. From the “Home” screen, the user has the 
option to hit any of the tabs or activities that we call: “Form”, “Map”, “Visual”, “Photo 
Gallery”, and “Help”. 
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Figure 6. Possible Lighthouse “Home” and “Help” Sereens ^ 


1, Form 

By clicking on the “Form” tab, the application would display the activity that 
allows users to enter new information collected. The view may be different on a mobile 
phone, tablet or laptop web browser to take advantage of the larger screen real estate. 
Figure 7 displays an example of the “Form” and “Map” activities. 

2, Map 

The “Map” tab would place drop pins on various locations on a map that 
reference different records in the database. Hovering over a drop pin would display a 
summary view of the information from the database that corresponds to that location. 
Location data could be provided by Google Maps when network connectivity is not an 
issue, but could prove impossible when there is no access to the Internet. Lack of 
connectivity could cause delays or other errors when running the application as it 

^ Figures 6, 7, and 8 were created by Mike Stevens of the CORE Lab after discussions with members 
of the Lighthouse working group. The photo gallery was modified by Marisol Torres to show images from 
http://vectorcharacters.net. 
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attempts to update the maps. Leaflet is a free, open-source JavaScript library that could 
be used to provide interactive maps for mobile devices (Leaflet 2014). 



Figure 7. Possible “Form” and “Map” Screens 


3, Visual 

One of the applications under the umbrella of Lighthouse is a tool that can take 
data of a specified format, aggregate that data and develop a link diagram. The li nk 
diagram is not an analysis of the actual data. Rather, the tool simply scans the data and 
tallies the number of times a piece of data is referenced and creates a visual 
representation of the data. The “Visual” tab is one mechanism that agency leaders agree 
would be needed to incentivize data collection. A collector can visually see what 
information is lacking in content. Based on their mission, collectors can choose to collect 
more information about those areas with only a few links, or choose to collect even more 
information on an area with a large number of li nk s. 

Since current operations require Internet access to send the information to a 
central database to be aggregated, reports and graphical representations of the data must 
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be done externally and downloaded on the mobile device when completed in order for 
collectors to view these diagrams. The ability for collectors and decision makers to do 
this locally from any remote location would be a major improvement. Figure 8 depicts 
example “Visual” and “Photo Gallery” screens. The Java Universal Network/Graph 
Framework (JUNG) is a framework written in Java that could be used for graphing and 
visualization on the server side and downloaded to mobile devices. MindFusion, 
described later, could be used to create the visualizations directly on the mobile device. 
The Photo Gallery activity would display every photo from the database. Clicking on a 
photo could provide several options to the user, from displaying information about that 
person, to creating new li nks or relationships between people. 
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Figure 8. Possible “Visual” and “Photo Gallery” Screens 


4, Photo Gallery 

The “Photo Gallery” is another capability that the working group users agreed 
would be extremely beneficial. The user could have several views based on the amount of 
screen real estate available. For example, on a tablet the screen could display nine or 
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twelve photos while the maximum number of photos that appear on a mobile phone could 
be four or six. A list view would show three photos with a list of details to the right of the 
photo. The user could also select a view with one photo that would zoom in on the photo, 
but also display the rest of the data from the database pertaining to that person. Hovering 
over a photo could display a summary of the information in the database, while clicking 
on a photo would take the collector to a “Baseball Card”. 

5, Help / Codebook 

The “Help / Codebook” screen provides the user the ability to learn more about 
how the application functions. In addition to explaining useful tips for the data collector, 
the codebook has definitions and explanations of terms used within the system, as well as 
the commander's intent of the actual collection activity. Upon clicking on this tab, the 
collector would be brought to the portion of the codebook that explains information about 
the last activity. For example, if the user were in the “Photo Gallery”, hitting the “Help” 
tab would display helpful information about the “Photo Gallery”. It is a continuous, 
scrollable view of the entire codebook. Figure 6 displays a sample “Help / Codebook” 
screen. 


6, Other Capabilities 

a. Baseball Card 

The Baseball Card is a view that collectors could get to from the “Photo Gallery”. 
This screen displays what is referred to as a baseball card because it is basically a 
summary of data from the database, similar to information that one might find on an 
actual baseball card of a major league baseball player. The view is a quick view for 
collectors to familiarize themselves with a specific individual. Double clicking on the 
photo in the gallery would take the collector to a screen displaying more information 
from the database about the person selected (similar to Figure 9 below). From that screen, 
the data collector could modify any of the existing data or simply scroll through the rest 
of the database information about that person. 
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Figure 9. Possible “Baseball Card” View (after Vectorcharacters 2014) 


b. New Links 

Working group members agreed that the ability to create new links or 
relationships between people or other information within the database would be 
extremely beneficial to Lighthouse collectors. Adding a relationship could be done in 
several ways, but the most simple would be from the Photo Gallery. For example, a long 
click on a photo could highlight a photo, placing the user in an “Add relationship” mode. 
A second long click on a different photo would open another screen where the user would 
enter information about the type of relationship being added. For example, it could show 
a screen with the photo or name of person 1 “is a “ [kinship type] ” “of ” photo or name 
of person 2, where [kinship type] is a dropdown menu from which the collector would 
select the type of kinship like cousin. If the type of relationship is new, the user could add 
a new one like “ conducts business with” or “purchases goods from” by selecting new 
from the drop down menu. 
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c. Export Capability 

The “Export” capability is a tool that working group members thought might 
make integrating with more robust social network analysis tools easier. This tool would 
enable collectors to create files from database data in specific file formats. For example, 
the collector could create .dot, .csv, .txt, or .xml files from the data in the local database. 
They could then send these files to other users in the area via NFC, Bluetooth, or Wi-Fi 
Direct. Additionally, the files could be sent to remote, external servers when Internet 
connectivity is established. 

d. Database Integration 

Another critical piece to make integration seamless is creation of a local database 
that can actually compare and share information with other local databases. This function 
would prove extremely powerful. Since the current implementation creates separate 
tables that do not interact with one another, all information is sent to an external, 
centralized database for any and all analysis to be performed. Reports are sent back to, or 
downloaded by collectors when connectivity exists. Smart phones and other mobile 
devices have processing capabilities that make it possible for them to perform some 
meaningful analysis locally, rather than using up valuable time and network bandwidth to 
send the information over the network first. 
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IV. METHODOLOGY 


The database arehiteeture deseribed in this ehapter does not capture information 
that may be required for the devices to know the authorized users or other authorized 
devices with which they can communicate. Additional information, perhaps stored in 
another table, is required for the mobile devices to establish the connections required to 
keep their data in sync with one another and/or the main database. The specific 
information required for communication depends on the communication technologies or 
protocols being used. We discussed some of the considerations for Wi-Fi Direct and 
Bluetooth information sharing in Chapter II, Sections B and C. We discuss two different 
methods for sharing data below. 

A. UPDATE USING FILE EXPORTS 

When synchronizing data between multiple devices, several approaches can be 
used. To explain the algorithm, we limit the number of devices to four. The algorithm, 
however, is designed to work with an arbitrarily large number of devices. 

SQLite databases on phones are not as robust as normal, full-featured SQL 
servers. While phones can support multiple tables in an SQLite database, as well as 
multiple databases on a phone, there are several limitations with mobile versions of 
SQLite. 

One major difference between SQLite and SQL implementations is that an 
SQLite database can only store three types of data: String, long and double. Normal SQL 
databases can support many different data types to include Boolean values, binary large 
objects (BLOBs), variable length character strings (varchars), and several more. To 
display data types not supported on SQLite, they must be converted properly and then 
stored locally. They can be reverted back to the appropriate format when being sent to 
other SQL databases, but that additional conversion may or may not be required. In fact, 
it may prove to be inefficient to constantly change the file format, but that really just 
depends on how the external database is being used and what other devices communicate 
with its database directly. There are APIs that enable developers to do the file 
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conversions with relative ease. For example, there are Java utilities that allow storage of 
BLOBs in SQLite databases on phones. Those BLOBs ean hold photos or other 
information not normally reeognizable by SQLite databases. Sinee SQLite databases are 
loosely typed, that is, do not restriet the actual data that is entered into a field, developers 
are responsible for ensuring the integrity of the database (Muraeh 2013). This makes it 
easier to use other data types inside of SQLite databases. 

Another limitation of the SQLite databases on a mobile deviee is direet 
eonneetivity to a database. The connection from one mobile device direetly to an SQLite 
database residing on a different mobile deviee is not permitted. The main reason for this 
restriction is that external devices would require aceess to the other deviees' file strueture, 
and permissions to aeeess, or modify files on an external deviee. This eapability is 
intrinsieally in violation of basie seeurity protoeols. Generally, no one wants external 
deviees to be able to eonneet to files on loeal deviees, mueh less modify them. They may 
want to be able to share a file, or obtain a file from another mobile deviee, but generally 
do not want their own files to be modified. Developers have had to come up with ereative 
ways to implement this capability. 

With this in mind, there are several routes that applieation developers ean take to 
make an applieation seem like it is synehronizing direetly. In the method deseribed here, 
it does not matter whieh teehnology is used to eonneet and send a file. All that matters is 
that a mobile device must be able to send a file to another device. Basically, any 
application that attempts to store data into a loeal database, must also be able to send that 
data to a file which would be shared with another mobile deviee. The next steps happen 
behind the scenes and are not observed by a easual user of the deviee. The following 
deseribes how four mobile deviees eould perform the synehronization in a baekground 
proeess without regard to the eonneetion teehnology. 

1. Use Case 

A team of four data eolleetors report to a village to obtain information about the 
loeal population. There is no existing network infrastrueture but the team leader did 
obtain the download from the master database prior to arriving at the village. He shares 
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the data with the other devices by using a specialized Bluetooth, NFC or Wi-Fi direct 
application that places the data in a specified location on each mobile device. Again, 
since complete files can be transferred via each of these methods, the specific connection 
method does not really matter at this point. 

A separate application is opened on each device that parses through the shared 
file, creates the appropriate local tables and allows the other data collectors to query or 
modify the stored data. This synchronization method uses a temporary database 
containing any of the modifications the user makes to the database. If the user never 
obtained the master file, he or she would still be able to collect new information. The 
team leader manually sets the device clocks. The application to collect information is 
started on each device by the individual collectors. Starting that collection application 
triggers the synchronization application, which runs in the background so it requires no 
additional action from the information collectors. 

All devices in use require uniquely identifiable names and each device has the 
names of other valid devices stored in local tables. The names could be the MAC 
addresses of the mobile device but for our example, we will use Ml, M2, M3, and M4, 
respectively. Each device with data to share, or modifications to the database would 
create a file using a standardized naming convention such as m2_dbname_l.csv. The first 
part of the name would correspond to the actual machine name of the device that created 
the file. The second part of the file name would be the database name for which the data 
belongs and the last part is an index. The index would iterate from 1 to 1000 and repeat. 
This is merely a precautionary measure to ensure that a device does not overwrite a 
previously written file that has not yet been aggregated and properly uploaded to the 
master database. The file could be a .txt, .csv, or .xml file. It really just depends on how 
the file is being accessed when it is time for synchronization to occur. 

This method of synchronization requires that one device is designated as the 

database master. This is for simplification since true peer-to-peer systems that allow 

every device to behave as a master can be extremely complex and difficult to implement. 

This adds several levels of complexity to ensure that data is aggregated properly, 

duplicated data is not added erroneously and multiple devices do not take multiple 
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attempts to update the same exact data, at the same time. This algorithm avoids those 
complications by establishing one as the leader or “master”. We name Ml as the device 
that will actually aggregate all data collected and eventually connect to the external 
database to upload all of the aggregated data. 

Each device will store data in a local database as well as create a .csv file in a 
specified location containing any data that has been modified or added to the database. 
The new and modified records will have their modified bit set as well as the emp id and 
timestamp for either the create dt or modified dt. Additionally, any records that should 
be deleted will have a delete flag set and record the timestamp into the modified_dt and 
emp id of the user dictating the change to be made. 

When a connection is established, the master device will check the specified 
location for files with the specific naming convention mentioned earlier. If there are files 
matching that naming convention, it will check for a semaphore file. The semaphore is an 
empty file following a similar naming convention except the file extension is .del or .ack 
extension. The .del informs the device that the file was aggregated and uploaded 
properly. The .ack informs the local device that a specific file can be deleted. To continue 
with this example, if m2_dbname_l.del exists. Ml knows that the information has 
already been aggregated. It deletes the file original .csv file but leaves the .del file as an 
acknowledgement to M2 that its data were received, aggregated and uploaded to the 
master database. The next time that M2 connects, if it detects an m2_dbname_l .del file, it 
knows that those changes were uploaded properly and can remove those records from its 
temporary database and commit those changes to its local database. M2 would then create 
an acknowledgement file with the same name and .ack extension. This is a semaphore 
that informs the local device that other files with the same name can be deleted. The 
temporary database would still contain all of the new data that was collected between 
pervious synchronization attempts. 

Since we established that Ml would be the local “master” responsible for 
aggregating data collected in the field, the other devices would not pull new data unless 
Ml specified that new data exists. So, if no device has changes to be made, no files will 

be created or shared. However, if there were a change. Ml would inform its peers that a 
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database update exists after a speeified amount of time or drop the most eurrent version 
of data into the appropriate loeal loeation at the toueh of a button. When those deviees 
want to obtain updated data, they would only obtain aetual updates from Ml. Although 
they would still obtain eopies of every deviee’s .esv, .aek and .del fdes, they would not 
aetually use any of the data in those files. Those files are simply replieated to ensure that 
Ml eould obtain MS’s update from M2 or M4 in the even that MS could not establish a 
connection to Ml directly. This method prevents loss of data in the event that network 
connectivity is nonexistent. 

This method is not very different than the current implementation of Lighthouse 
in that updated information is gathered and used to generate reports that are stored on the 
mobile device. These files are stored in an “outbox” for use when connectivity is not an 
issue. So, when there exists a connection to the Internet, the “outbox” transmits the 
messages that have not been sent. 

This is not truly a departure from current operations. The only benefit of this 
capability would be that users could share their information with one another so that in 
the event that one of their devices is compromised or lost, the information collected on 
that device is not lost. Rather, it is on another, local mobile device that can send the files 
to the cloud when connectivity is restored. While useful, this extension has limited 
benefit. 

It is more useful to provide this capability as an important adjustment is made to 
the application's current capability. The current application must include a reader that can 
parse through the files sent by other users and stores the information in each device's own 
local database. Additionally, the new generation would have to support additional 
capabilities such as visualization and photo gallery search to be truly useful. This would 
mean the collectors could potentially benefit from the information that was collected by 
others. Absent these, the data would still need to be uploaded to the cloud for 
exploitation. 
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B, 


DATABASE SYNCHRONIZATION 


If we want the information eolleeted by users to be shared between eolleetors 
more readily so that the information eould be used by the eolleetors on the ground, the 
previously described implementation would not be enough. 

In the following approach, instead of storing and sending actual fdes, we share 
individual objects or records. In order for this approach to work, there are several key 
requirements that need to be satisfied. 

The first requirement is that the system uses one of the three communications 
mechanisms described in Chapter III, Section B2. Each of these connection mechanisms 
has specific requirements in order to share data. Bluetooth requires MAC addresses while 
Wi-Fi Direct requires IP address. These details are not critical for this research since IP 
addresses can be assigned by group leaders upon creation of a group. At a minimum, all 
devices must know the device name and MAC address of all devices to which it must 
send information. Although this is not a requirement when using Bluetooth technology, 
as mobile devices truly need not know to whom they wish to connect in advance. This is 
a requirement since developers and users wish to limit information leaks to unauthorized 
devices. This acts as an access list, storing specific information that would make 
connecting to a device more rapid. Other information would be stored to ensure 
synchronization functions are executed properly. 

1. Data Sharing Algorithm 

The algorithm below describes an implementation that would work for sharing 
data. It would ensure that new and modified data is only sent to other devices when 
certain conditions exist, preventing unnecessary traffic. Additionally, it prevents against 
out of sync databases by attempting to send updates at least a specific number of times 
(three in our case) unless it receives an acknowledgement that data was successfully 
received. 

When a device has data to send, it starts counting from zero. It uses a sent and ack 

array to keep track of which devices have yet to receive the data. If the sent flag is 0, the 

device will attempt to send the data and set the sent flag to one after the data has been 
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successfully sent. If the sent flag is 1, it moves on to the next deviee awaiting the data. 
After going through eaeh device in the deviee list, it resets the sent array, inerements the 
number of attempts and starts going through the deviee list again. After the third attempt 
to send, no further attempts to send that data are made. Pseudo eode for the algorithm is 
provided below. 


Each device will perform the following steps to send: 

Step 1. Create devList = DL[ ] - DL[ i ] ; local device = DL[ i ]; 

Step 2. Initialize arrays and variables: 
n= I DL[ ] I; 
sent,[ ] = 0; 
ack,[ ] = 0; 
ree/[ ] = 0; 
attempts = 0; 
maxTimes = 3; 

Step 3. Gather data to send: 

DATA = getData() { 
check messages table { 

if old messages still unsent, re-send 

} 

results = query( ) ; //select * from table where modified == “Y”; 
if (results == null or “ “ ) { 

EXIT // do not continue; wait until there is something to send 

} 

else { 

mid = getMessageID(); // get next available message id 
return newData = DL[ i ] + “, “ mid + “: “ results ; 

} 

} // end of getData( ) 

Step 4. While ( attempts < maxTimes ) { 
for (i in devList ) { 
if(aek[/] != 1 ) { 
if ( sent[ / ] == 0 ) { 

sendData( devList[ i ], DATA) { 

**here, i is the destination. 

**DATA contains the source, mid and data updates 

}; 

sent[ / ] = 1; 

} // end sent[ i ] cheek 
} // end if ack[ i ] eheck 
next i; 

} // end iov-dev loop 
attempts++; 
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} // end while 


2, Receiving Data Algorithm 

The algorithm below deseribes an implementation that would be used when 
reeeiving data from an authorized deviee. It would store the data reeeived in its loeal 
SQLite database and respond with an aeknowledgement to the sender. The sent and aek 
arrays maintain the statuses of the sent and aek flags respectively and are the same arrays 
mentioned in the sharing algorithm. The acknowledgement is an effort to prevent 
unnecessary re-transmission of data. When a device has received data from an external 
device, the device sends an aek containing the message ID to the sender of that message. 
Upon receipt of an aek, the message sender sets the aek flag and no longer attempts to 
send that message to the device for which the aek was received. Pseudo code for the 
receiving algorithm is provided below. 


Each device will perform the following steps when receiving data: 

Recall from earlier that D,[ ] = DL[ ] - DL[ i ] 

Step 1. RECEIVED = recData() { 

source = DATA[ 0 ]; m id = DATA[ 1 ]; newData = DATA[ 3 ]; 
determine message type 
if type == aek , 
return aek 

else newData == (string of records; 

comma separated columns; 
records separated by \n) 

return newData; 

}; // What was received — (aek or newData ) 

Step 2. idx = getlndex( ); // get position in array for device that sent the data 
Step 3. switch ( RECEIVED ) 

{ 

case “aek” : 

ack,yx[ devEist[ idx\ ] = 1 
break; 
default: 

updateEocalDB() { 

update = sql code that enters information received into local database 
if record already exists, check create date and modified date 
save the newer record; 
if questionable, save to error table. 

set success to true if update successful, else success = false 
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}; 

if ( success ) { 

sendAck( devList[/Jx], m_Id) { 

send from my deviee to devList[idx] 

— dev, mid stored 

}; 

} 

}; 

3, Algorithm Walk-Thru 

As one reviews the algorithm, one should notiee that it does not specify a specific 
action that triggers or starts the sharing. This was done intentionally. The trigger could be 
based on time, a certain number of ehanges or new reeords added to the local database, a 
button pressed by the user, or other aetion specified by the developers or deeision makers. 
Additionally, this researeh focused on an algorithm that would be as generie as possible 
so the method of eonneetivity (Wi-Fi Direct, Bluetooth, WAN, LAN, etc.) was not 
important either. 

As an example, the data sharing and reeeiving programs ean be separate activities 
triggered by loading a data eollection form. These aetivities ean be separate applieations, 
or part of the data colleetion application. In this mode, these would be running in the 
baekground requiring no action from the data eolleetor. 

As mentioned previously, after a eertain amount of time, or after colleeting a 
certain amount of records, the sharing program verifies that there is data that to be shared. 
It traeks whether data needs to be sent by eheeking the sent and aek arrays. This is not the 
only way this algorithm could work. It could also use a separate message table that holds 
the message IDs of all messages sent and reeeived. Rather than eheeking the position in 
the sent and ack arrays, it would cheek the field within the message table eorresponding 
to the device. If the ack for a message ID for a speeifie device is not received, it 
retransmits that message to the speeifie device. In either ease, this is what oeeurs in the 
Prep DATA portion of the fiowehart. It will start the attempt eount and send data if an 
aek has not yet been reeeived. Upon sending data, it will set the sent flag eorresponding 
to destination deviee. Otherwise, it will move on to the next deviee in the deviee list until 
it has gone through all of the devices in the deviee list. 
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Figure 10. 

Sharing Data Flowchart 



The receiving flowchart below depicts the actions taken by the receive listener. 
The listener will begin running on the mobile device upon start up of the collection form. 
As in the previous explanation, there is no special action required of the user to start the 
listener. This program is running in the background unbeknownst to the user and does not 
inhibit the users’ ability to enter new data in any way. 

The device is listening for messages from any of the devices in its device list 
array. When it receives a message, it checks the source of the message. The sharing 
program uses this information to identify the index in the array that will be modified. 
Then, it starts to parse the message received to verify if the message is actual data or an 
acknowledgement. If it is an acknowledgement, the index for the source device in the ack 
array is set to 1 and the device goes back to listening for data. 

If the message is an actual message, it is parsed and the local database is updated. 
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Figure 11. Receiving Data Flowchart 


4. Implementation 

The prototype developed for this demonstration was created using a free Android 
Open Source Project Bluetooth Chat Application found on GitHub ^ The code was 
modified using the Android Developer Tools Build: v22.6.2-1085508, which includes the 
Eclipse Platform. The application was tested on Android devices with API Level 15 (Ice 
Cream Sandwich) or higher. This was to ensure stability and consistency of the 
application while using the local SQLite database and Bluetooth technology 
simultaneously, as well as ensuring compatibility within Eclipse. 

The discussion below walks through the above algorithm and shows three 
devices. Table 1 lists the devices used, their device names, operating systems and 
database version. 


^ According to GitHub.com, their website was founded by Tom Preston-Wemer, Chris Wanstrath, and 
PJ Heyett and is one of the most popular code hosting/sharing websites in use. 
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DEVICE # 

DEVICE NAME 

TYPE 

DESC 

Database 

I 

NPS5 

GT-19300 

Android v4.03 

SQEite 

2 

NPSI 

Nexus 7 

Android v4.03 

SQEite 

3 

Galaxy Nexus 

Galaxy Nexus 

Android v4.03 

SQEite 


Table 1. List of Devices Tested 


The application starts off by checking whether Bluetooth is on or off. If it is off, 
the user is prompted to turn on Bluetooth as depicted by the Program Action Sequence in 
Figures 12, 13 and 14. 



>00 



Gender 


Bluetooth pcrmtssion request 


Application is requesting 
permission to turn on Bluetooth 
AlloW> 




t\ % Mhi 4 Ml 


Compile Data 



Figure 12. Start of Program Action Sequence of NPS5 


If Bluetooth is on, it opens the data entry form with example text so the user 

knows the correct format of each field. This application forces the user to scan for 

devices manually, however, this capability can be programmed to execute in the 

background. The intent is for the device to make connections automatically in the 

background without requiring any action by the collector. In this way, the collector would 
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simply continue to collect data, without having the additional burden of sharing the 
information, or verifying that data is being shared. This application forces the actions to 
be done manually in order to demonstrate the different actions that take place to establish 
a connection with nearby data collectors. 

The prototype discussed here does not trigger the algorithm described. Rather, it 
simulates similar actions. For example, it forces the user to manually connect to an 
external device prior to entering any records into the database. In actuality, the 
connections need not be established upon startup of the application. This was a design 
choice to maintain the chat functionality. The interface allows the user to enter one 
record. When the user hits the compile button, the data is prepared for sending. The user 
reviews the information being sent at the bottom of the screen and must hit the send 
button to send the data. If for any reason the connection to the external device is lost, the 
user must re-establish connectivity prior to sending any data. On the receiving end, the 
received message is stored in the device's local database. All of the required actions occur 
manually in this prototype. The actual application would run behind the scenes. 





FName Mickey, I Name Mouse, 
Gender M, DOB: 19281118 


Figure 13. Steps 3 and 4 of NPS5 
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CT-19300 



Inserting into database:. Message ID: cjd: 

Data; NPS5; FName; Mickey, LName: Mouse, 

Mf Gender M, DOB: 19281118 pe. 

Gender: M, DUB: 192811 IB 

Send 


)NLINE 360x640 (3.5 Mb) 1072 ms (3.3 Mb/s) 

Figure 14. Final Step of NPS5 

Stepping through the program action sequence above, we see the various screens 
seen by a user, NPS5, who is already connected to an external device, NPSl. Data is 
entered into the collection form, compiled, displayed at the bottom of the screen and sent 
to the external device. After hitting the send button, the information is entered into the 
local SQLite database and sent to the external device. 

Figure 15 and 16 shows the screens viewed by NPSl. Starting from the top left, 
we see that this tablet is connected to an external device named NPS5. The screen on the 
top right shows that it received data from NPS5 and displays the information received. 
There is no photograph of the message stating that the information has been stored onto 
this device's local SQLite database because even with a long toast message, I was not 
able to perform the screen capture quickly enough. 
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Figure 15. Start of Program Action Sequence for NPS1 


In order to demonstrate that data is sent in both directions, next, NPSl enters 
information on the top of Figure 16 and reviews it on the bottom of the tablet after hitting 
the “Compile Data” button. Upon hitting the send button, we see the updated screen with 
both records displayed in the photo to the bottom of Figure 16. 
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Figure 16. Screens 3 and 4 of NFS 1 


Figure 17 shows the event as seen by the NPS5 device. 
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First Name: Mickey 
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Compile Data 


Me: NPS5: FName: Mickey, LName: Mouse, 
Gender: M, DOB: 19281118 

NPS1: FName: Mike, LName: Jones, 
Gender: M, DOB: 19901211 


360x640 (3.S Mb) 1051 ms (3.3 Mb/s) 


Figure 17. NPS5 screen after receiving data from NPSl 


After the NPS5 has finished sharing its information with NPSl, it checks its 
device list to see if there are other devices that still require the information. The device 
executes the algorithm once again to share with a third device called Galaxy Nexus. 
Figures 18 through 22 show the program sequence for NPS5 sharing its data with the 
third device, Galaxy Nexus. Again, although the prototype shares one record at a time, 
the actual application would execute a query to obtain all of the updated or new records 
and send all the records to the next device at the same time. The receiving device would 
parse through the message received and update its local database. 
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Figure 19. NPS5 receiving data from Galaxy Nexus 
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Figure 20. NPS5 after inserting data from Galaxy Nexus 
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Figure 21. Galaxy Nexus sharing data with NFS5 
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At this point, NPS5 has shared all of its information with the other two devices 
that required its updates. Next, the second and third devices would make the required 
connections to share their respective data. Figures 23 through 25 depict the actions taken 
by NPSl and Galaxy Nexus in order to share their data from the Galaxy Nexus 
perspective. Figures 26 through 28 show the same actions from the NPSl perspective. 
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Figure 24. Galaxy Nexus after sending data to NFS 1 


53 






Figure 25. Galaxy Nexus after reeeiving data from NPSl 
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Figure 27. NPSl sending data to Galaxy Nexus 
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After this iteration of the algorithm, the databases on all three devices have all 
three records that needed to be shared. To show that all the records are on each device, a 
separate database reader was used to display the records of the database. Figures 29, 30, 
and 31 show the entries in each local database. 
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Figure 29. NPS5 Database Records 
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C. DATABASE STRUCTURE 

Figure 32 depicts a sample database structure for a centralized database. Its 
format is slightly different from the SQLite database on a mobile device. 
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Figure 32. Sample SQLite Data Structure 
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1. Sample Local SQLite Database Tables 

The tables below represent data deseriptions of possible SQLite database tables 
for the prototype ereated. These ean be exaetly the same as the external or eentralized 
database, with minor differenees in the data types of the fields. All tables in the eentral 
database need not be represented in the tables on the loeal deviees and viee versa. That is, 
only required fields need to be refereneed. This saves spaee on the mobile deviees and 
improves proeessing times beeause developers need not alloeate spaee and resourees for 
unused data. 


TITLE 

TYPE 

DESC 

empid 

text 

Unique ID eonsisting of first and last initials of information 
eolleetor with 4 digits appended to end. Must be unique to 
identify individual eolleetor. 

Iname 

text 

East name of information eolleetor. 

finame 

text 

Eirst name of information eolleetor. 

ereatedt 

number 

Date information eolleetor was added to employee table. 


Table 2. Emp Table 


TITLE 

TYPE 

DESC 

personid 

text 

Unique primary key. UserlD of ereator with pseudorandom 
number appended to it that uniquely identifies the reeord. 

Iname 

text 

East name of individual whose information is being eolleeted. 

finame 

text 

Eirst name of individual whose information is being eolleeted. 

gender 

text 

One eharaeter field representing the gender of the person. 

dob 

number 

Six digit number representing the date of birth of the 
individual. 

ereatedt 

number 

Twelve digit number representing the Date/time stamp when 
reeord was ereated. 

ereator id 

text 

UserlD of person who ereated the reeord 

modified 

text 

Y or N indieating whether reeord has been modified sinee 
pulled from eentral database. 

modifieddt 

number 

Twelve digit number representing the Date/time stamp of 
modifieation to existing reeord. 

modifiedid 

text 

UserlD of last person to edit the reeord 


Tables. Sample Person Table 
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TITLE 

TYPE 

DESC 

personid 

text 

Eoreign key indicating person described by this data. 

datafield 

text 

Type of data belonging to person id. Ex cell_phone 

valuefield 

text 

Value of the data field. Ex. 8453452314 

created! 

number 

Date photo stored in table 

creator id 

text 

UserlD of person who created the record 


Table 4. Sample Person Data Table 


TITLE 

TYPE 

DESC 

personid 

text 

Eoreign key indicating person described by this data. 

photo 

blob 

Photo of person matching the person id. 

created! 

number 

Date photo stored in table 

creator id 

text 

UserlD of person who created the record 


Table 5. Sample Photo Table 


2. Sample Database Creation Script 


// create emp table 

// This table holds administrative data pertaining to the information collectors. 

// The emp id identifies information collector 

// that either created or modified a record in the database. 


CREATE TABLE EmpTable ( 

emp id VARCHAR2(6) NOT NULL CONSTRAINT EmpTable_pk PRIMARY KEY, 

Iname VARCHAR2(30), 

fname VARCHAR2(30), 

created! DATE 

); 


// create person table. This table contains data collected by employees. 

CREATE TABLE PersonTable ( 

person id VARCHAR2(30) NOT NULL CONSTRAINT PersonTable_pk PRIMARY KEY 

UNIQUE, 

Iname VARCHAR2(30), 

fname VARCHAR2(30), 

dob VARCHAR2(30), 


gender 

created! DATE, 

creatorid VARCHAR2(6), 

modified VARCHAR2(li 

modifieddt DATE, 

modifiedid VARCHAR2(6) 


); 
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// create person data table 

// This tables holds fields relating to corresponding person id in PersonTable. 

// These are data fields that a person could have more than one such as phone number, vehicle, cousin, sister, 
etc. 

// Storing this data in a separate table prevents from having to add new columns to any table in the database. 
CREATE TABLE PersonData ( 

person id VARCHAR2(30) NOT NULL CONSTRAINT PersonData fk FOREIGN KEY, 

datafield VARCHAR2(3oi 

value)field VARCHAR2(30), 

created! DATE, 

creatorid VARCHAR2(6) 

); 


// create photo table 

// This table can hold multiple photos of a specific person. Using a separate table enables us to store multiple 
photos. 

CREATE TABLE PhotoTable ( 

person id VARCHAR2(30) NOT NULL CONSTRAINT PhotoTable fk FOREIGN KEY, 

photo BLOB, 

created! DATE, 

creatorid VARCHAR2(6) 

); 


3. Chapter Summary 

When examining the sample SQL seripts, sereen captures and algorithms 
described in this chapter, one must recognize that subtle changes must be made to tie all 
of the pieces of this project together. 

The queries are for creating the actual centralized database rather than the code 
for creating the database on the local device. The local database on the local device was 
created using a datatable.java file within the android application. That code is not 
displayed because it varies depending on the SDKs and API being used. Personal 
preferences of developers play a significant role in how those tables are created as well. 

As mentioned at the start of this chapter, the database architecture described in 
this chapter does not capture information that may be required for the devices to know 
the authorized users or other authorized devices with which they can communicate. Since 
the information required for communication varies with the communication technology 
or protocol being used, this focused on an algorithm that does not require a specific 
technology. The prototype discussed demonstrates how sharing data between mobile 
devices can be implemented. 
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V. CONCLUSIONS 


A. MAJOR ACHIEVEMENTS 

The goal of this research was to identify and outline improvements that could be 
made to the current Lighthouse Suite of applications, specifically in how information 
could be shared among mobile devices in remote or challenged networking environments. 
To this end, research was done to identify challenges to implementing the recommended 
improvements and recommendations about which technologies could be levied in order 
to make sharing data in these environments more robust and transparent to the data 
collectors. 

The research described the various products within the current suite of 
applications and how some of those products could be consolidated to provide better 
fidelity, improve sharing time, and possibly cut down on the overall analysis time. The 
most significant contribution of this research is an algorithm for database synchronization 
that would ensure data synchronization among devices on a local network, even when 
connectivity to cloud or Internet services is not available. 

It involved the modification of an existing Android application created for 
Bluetooth Chat that allows a collector to store new information in their local database and 
share that information with known users within Bluetooth range. Although this prototype 
is fairly simplistic, it demonstrates that data collectors can collect data in these remote 
areas and share that data with other users of the system, without requiring actual Internet 
connectivity. Additionally, it walks through the algorithm and prototype to show that 
both function correctly and identifies areas that could be improved. 

B. FUTURE WORK 

1. Seamlessness 

This solution used multiple applications in order to demonstrate a small portion of 
the capabilities required for the next generation of Lighthouse. Developing the entire 
application is a large task, requiring several engineers. A team of developers would be 
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required for each piece of Lighthouse to be developed in a timely fashion. Additionally, 
for users to move properly from one activity to another, developers must be able to focus 
on specific aspects of the program. Modularity in the development of the application 
would allow experts in specific areas to focus on their piece of the puzzle. For example, 
some developers may have more expertise in map development or database management, 
while another may be more proficient in the visualization APIs. 

2, Connectivity 

Lighthouse can continue to be improved in a number of ways. This solution was 
developed using Bluetooth technology. More research needs to be done using Wi-Fi 
Direct as a means of connectivity instead. Wi-Fi Direct could mean that collectors could 
share information at greater distances. This capability is more powerful than the 
Bluetooth solution, not just because of the greater distance, but because the capacity to 
maintain constant connections between multiple devices is greater when using the Wi-Fi 
Direct technology. 

Additionally, data could be broadcasted to all devices simultaneously using local 
IP addresses assigned to each device when connectivity is requested, rather than the 
pairwise sharing used with Bluetooth connectivity. This would improve the overall 
synchronization time as well. 

3, Off-line Maps 

According to their website. Leaflet is a JavaScript Library used to create 
interactive, mobile device “friendly” maps. It is an open source library located on GitHub 
developed by Vladimir Agafonkin and other “contributors” (Leaflet 2014). While 
GoogleMaps could be used when Internet connectivity exists, it is extremely difficult to 
use without connectivity. The beauty of Leaflet is the Lighthouse developers would have 
total control over the map tiles used. The appropriate map tiles could be loaded onto the 
mobile devices when loading the “mission” data. Additionally, the application could 
access the appropriate map tiles even without connectivity to the Internet. Figure 33 
depicts a sample Leaflet map view with a blue drop pin in the center. 
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Figure 33. Sample Map View Using Leaflet (from Leaflet 2014) 


4, Data Visualization 

MindFusion Diagramming Library for Android is an Android Java Class Library 
that developers can use for providing a means for their users to visualize data. The API 
enables users to view graphs, flowcharts, genealogy trees and more (Mindfusion 2014). 
Figure 34 shows two possible views that would prove helpful to Lighthouse data 
collectors, depending on the goals of the collectors. The ability to create these views 
would enhance operations because decisions could be made regarding collection 
priorities at the local site rather than sending the data to a central database via the Internet 
and awaiting results of the analysis. This functionality would decrease the amount of time 
it takes for leaders at the remote site to obtain mission critical information, enhancing 
decision making capabilities. 
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Figure 34. Sample Link and Tree Diagram (from MindFusion 2014) 


5, Device Independence 

The original idea was to create the application using PhoneGap in order to 
develop a device independent application that could incorporate several tools used in 
Lighthouse. PhoneGap allows developers to use F1TML5 and JavaScript for creating 
device independent applications. Basically, the same code is used for desktop or laptop 
web browser, iOS device, or Android device. The prototype was scaled down after 
multiple challenges experienced due to the learning curve and shortened timeline. Using 
HTML5, it proved difficult to access the local device's file system. This made working 
with the local SQLite database a challenge. However, if having the data stored in an 
actual SQLite database is not critical, the data storage could be handled using either 
WebDB or IndexedDB. 

More research is required to test the capabilities of PhoneGap. Additional 
capabilities are being added to the PhoneGap libraries that enable access to specific 
sensors on mobile devices. If the APIs that enable access to the mobile device's Wi-Fi 
adapter are robust, a Wi-Fi Direct solution for connectivity could be developed. Using 


66 






















this, one would not have to use the loeal SQLite database of the mobile deviee. Instead, 
the developer would use either WebDB or IndexedDB. In either ease, sharing or sending 
data to an external server is fairly trivial using HTML5. 
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